Method and apparatus for remotely controlling home device in home network system

ABSTRACT

A method for remotely controlling a home device in a mobile terminal is provided. The method includes receiving a chat message input to the mobile terminal for controlling at least one home device through an instant messenger application installed in the mobile terminal, and sending the chat message to a messenger server. The sent chat message is converted into a control command, which is sent to the home device.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit under 35 U.S.C. § 119(a) of a Koreanpatent application filed on Sep. 5, 2013 in the Korean IntellectualProperty Office and assigned Serial No. 10-2013-0106547, the entiredisclosure of which is incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to a home network system. Moreparticularly, the present disclosure relates to a method and apparatusfor remotely controlling an operation of a home device using a mobileterminal

BACKGROUND

A home network system denotes a system that connects various homedevices installed in a home over a wired or wireless network to enablecommunication between the home devices.

The home network system may integrate home devices using a Home Gateway(HGW or H-GW), and connect the integrated home devices over an externalpublic data network (e.g., an Internet Protocol (IP) network) to providea desired service to a user who wants to control the home devices.

Recently, in line with the increasing diversity of home devices, theuser needs have been increasingly diversified. Therefore, there is aneed for efforts to provide the best service, accepting the various userneeds.

In recent years, various methods have been developed to remotely controlhome devices using a mobile terminal of a user.

In the existing method, a home device may be remotely controlled onlythrough a dedicated application agreed between the user and the homedevice. Therefore, the home device may not be remotely controlled,unless the dedicated application for controlling the home device isinstalled in the user's mobile terminal.

The above information is presented as background information only toassist with an understanding of the present disclosure. No determinationhas been made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the present disclosure.

SUMMARY

Aspects of the present disclosure are to address at least theabove-mentioned problems and/or disadvantages and to provide at leastthe advantages described below. Accordingly, an aspect of the presentdisclosure is to provide a method and apparatus for remotely controllinga home device using an instant messenger program universally installedin a mobile terminal, without installing a dedicated application forremotely controlling the home device in the mobile terminal.

In accordance with an aspect of the present disclosure, a method forremotely controlling a home device in a mobile terminal is provided. Themethod includes receiving a chat message input to the mobile terminalfor controlling at least one home device through an instant messengerapplication installed in the mobile terminal, and sending the chatmessage to a messenger server. The sent chat message may be convertedinto a control command, which is sent to the home device.

In accordance with another aspect of the present disclosure, a methodfor remotely controlling a home device in a messenger servercommunicating with a mobile terminal is provided. The method includesreceiving a chat message for controlling at least one home devicethrough an instant messenger application of the mobile terminal,converting the chat message into a control message, and sending theconverted control message to a home network server.

In accordance with further another aspect of the present disclosure, amobile terminal for remotely controlling a home device is provided. Themobile terminal includes a storage unit configured to store an executionfile of an instant messenger application, a controller configured toexecute the execution file of the instant messenger application, and toreceive a chat message input to the mobile terminal for controlling atleast one home device through the executed instant messengerapplication, and a communication module configured to send the chatmessage to a messenger server.

In accordance with yet another aspect of the present disclosure, amethod for remotely controlling at least one home device in a mobileterminal that includes a communication module configured to communicatewith a messenger server, a storage unit configured to store an executionfile of an instant messenger application, a controller configured toexecute and control the instant messenger application, and a displayconfigured to display an execution screen of the instant messengerapplication is provided. The method includes providing a chat listscreen including a home device list in response to execution of theinstant messenger application; providing a chat window for at least onehome device selected from the home device list, and sending a chatmessage entered through the chat window to the messenger server, forcontrol of the home device.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a block diagram illustrating a structure of a home networksystem according to an embodiment of the present disclosure;

FIG. 2 is a block diagram illustrating a configuration of a home networksystem accessible by a mobile terminal according to an embodiment of thepresent disclosure;

FIG. 3 is a block diagram illustrating a configuration of a home networkserver according to an embodiment of the present disclosure;

FIG. 4 is a block diagram illustrating a configuration of a messengerserver according to an embodiment of the present disclosure;

FIG. 5 is a block diagram illustrating a configuration of a mobileterminal according to an embodiment of the present disclosure;

FIG. 6 is a block diagram illustrating a configuration of a home networksystem for remotely controlling operations of home devices according toan embodiment of the present disclosure;

FIG. 7 is a block diagram illustrating a configuration of a home networksystem for remotely controlling operations of home devices using aninstant messenger according to an embodiment of the present disclosure;

FIG. 8 is a flow diagram illustrating a registration process ofregistering a home device in a home network server according to anembodiment of the present disclosure;

FIG. 9 is a flow diagram illustrating a process of registering a homedevice in a messenger server through an instant messenger applicationand a process of remotely controlling the registered home deviceaccording to an embodiment of the present disclosure;

FIGS. 10A, 10B, 10C 10D, 10E, 10F, and 10G illustrate examples ofvarious user interface screens displayed on a mobile terminal duringexecution of remote control according to an embodiment of the presentdisclosure; and

FIGS. 11A and 11B illustrate examples of user interface screensdisplayed on a mobile terminal during execution of remote controlaccording to another embodiment of the present disclosure.

Throughout the drawings, like reference numerals will be understood torefer to like parts, components, and structures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the present disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skilled in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thepresent disclosure. In addition, descriptions of well-known functionsand constructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of the presentdisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of the presentdisclosure is provided for illustration purpose only and not for thepurpose of limiting the present disclosure as defined by the appendedclaims and their equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

By the term “substantially” it is meant that the recited characteristic,parameter, or value need not be achieved exactly, but that deviations orvariations, including for example, tolerances, measurement error,measurement accuracy limitations and other factors known to those ofskill in the art, may occur in amounts that do not preclude the effectthe characteristic was intended to provide.

An embodiment of the present disclosure will now be described in detailwith reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating a structure of a home networksystem according to an embodiment of the present disclosure.

Referring to FIG. 1, the home network system may include home devices110 each having control and communication functions, a home networkserver (or a home gateway) 120, and a communication network (orInternet) 130. The home devices 110 may include smart appliances 112,security devices 114, lighting devices 116 and energy devices 118, allof which are located in the home. For example, the smart appliances 112may include a refrigerator, a washer, a robotic vacuum cleaner and thelike. The security devices 114 may include a door lock, a securitycamera, a security sensor and the like. The lighting devices 116 mayinclude a fluorescent lighting fixture, a Light Emitting Diode (LED)lighting fixture, and the like. The energy devices 118 may include apower meter, a power socket, an electrical outlet, a multi-tap, and thelike. In addition, the home devices 110 may include a Personal Computer(PC), an Internet Protocol (IP) camera, an Internet phone, awired/wireless phone, a home mobile phone, and the like.

The home devices 110 may be configured to communicate with the homenetwork server 120 in accordance with a wired/wireless communicationscheme, receive a control message sent from the home network server 120,and transmit requested information to the home network server 120.

The home network server 120 may include communication modules forcommunicating with the home devices 110 in accordance with thewired/wireless communication scheme, a storage unit for registering andstoring information about the home devices 110, and a controller forcontrolling operations and states of the home devices 110 and collectingand managing necessary information from the home devices 110. The homenetwork server 120 will be described in detail below with reference toFIG. 4.

FIG. 2 is a block diagram illustrating a configuration of a home networksystem accessible by a mobile terminal according to an embodiment of thepresent disclosure.

Referring to FIG. 2, the home devices 110 (110-1 to 110-n) in the homenetwork system may be connected to the home network server 120 inaccordance with the wired/wireless communication scheme. The homenetwork server 120 may access a wireless network 130 b or a wirednetwork 130 a such as the Internet.

The home network server 120 may allow access to a messenger server 200(which is not located in the home) over the communication network 130(including the wired network 130 a and the wireless network 130 b). Thehome network server 120 may send a control message received from themessenger server 200 to a relevant home device individually, or may sendthe control message to grouped home devices at the same time, forsimultaneous control of the grouped home devices.

The communication network 130 may include at least one of an IP networksuch as the Internet, and a wireless network that a mobile terminal canaccess.

The home network system shown in FIG. 1 may provide, over thecommunication network 130, a home entertainment service such as InternetTeleVision (TV) (e.g., IPTV) and Video on Demand (VoD), a home datacommunication service such as data sharing, Voice over IP (VoIP) andvideo phone, and a home automation service such as remote control ofhome appliances, remote meter reading, and security and emergencyservices. In other words, the home network system in FIG. 1 may connectall types of home devices used inside and outside the home over onenetwork to control them. Therefore, the term ‘in or inside the home’ asused herein may be construed to include even ‘outside the home’ as longas a home device located outside the home can be connected over onenetwork and controlled.

A user may remotely control the home network server 120 installed in thehome network system, or may remotely control each home device 110 overthe home network server 120, using an instant messenger application (ora messenger application) installed in his/her external communicationterminal (e.g., wireless communication equipment such a mobileterminal). The mobile terminal may include, for example, a PersonalDigital Assistant (PDA), a smart phone, a cellular phone, a tabletcomputer, a laptop computer and the like, all of which have acommunication function. The mobile terminal may access the home networksystem over the Internet and the operator network.

A mobile terminal 300 may access the wireless network 130 b thatincludes a wireless access network and an operator core network. Themobile terminal 300 may access the messenger server 200 via the wirelessnetwork 130 b. The wireless network 130 b may include, for example, a2^(nd) Generation (2G) or 3^(rd) Generation (3G) cellular communicationsystem, a 4^(th) Generation (4G) communication system, a Long-TermEvolution (LTE) communication system, a World Interoperability forMicrowave Access (WiMAX) communication system, and the like. A detaileddescription of the mobile terminal 300 will be made below with referenceto FIG. 5.

The messenger server 200 may access the home network server 120 via thewired network 130 a, and access the mobile terminal 300 via the wirelessnetwork 130 b.

An instant messenger application may be installed in the mobile terminal300 by the user. Alternatively, the instant messenger application may beinstalled in advance before the user purchases the mobile terminal 300.Using the instant messenger application, the user may separately controlan operation of the home network server 120 or operations of the homedevices 110 connected to the home network server 120, or may control, incommon, operations of grouped home devices.

If the messenger application is executed, a control command in the formof a chat message may be sent to the messenger server 200 from themobile terminal 300 in response to a user input. The messenger server200 may convert the control command in the form of a chat message into acontrol message agreed between the user and the home network server 120,and send the converted control message to the home network server 120.The home network server 120 may check identification information andcontrol command for a home device, which are included in the receivedcontrol message, and control an operation of the identified home device110 in response to the control command.

FIG. 3 is a block diagram illustrating a configuration of a home networkserver according to an embodiment of the present disclosure.

Referring to FIG. 3, the home network server 120 may be connected to atleast one of home devices 110-1 to 110-n using at least one ofcommunication modules 122 to 126. The communication modules 122 to 126may include at least one of a Wireless Local Area Network (WLAN) module122 (e.g., Wireless Fidelity (WiFi)), a ZigBee module 123, a Bluetoothmodule 124, a Near-Field Communication (NFC) module 125, and a wiredcommunication module 126. In addition, the home network server 120 mayfurther include at least one of a controller 121, a network interfacemodule 127, a User Interface (UI) 128, and a storage unit 129.

The controller 121 may include at least one of a Central Processing Unit(CPU) 121-1, a Read-Only Memory (ROM) 121-2 storing a control programfor control of the home network server (or the home gateway) 120, and aRandom Access Memory (RAM) 121-3 that is used as a memory area foroperations performed in the home network server 120. The controller 121may communicate with the home devices 110-1 to 110-n through thecommunication modules 122 to 126 in response to execution of programsstored in the ROM 121-2 or the RAM 121-3, or of application programsstored in the storage unit 129. The controller 121 may generate acontrol command that is sent to the home devices 110-1 to 110-n, or maystore information collected from the home devices 110-1 to 110-n in thestorage unit 129.

The user interface 128 may include an output module (not shown) such asa display, a speaker and an alarm lamp, and an input module (not shown)such as a touch screen, a keypad and a microphone. The user may use theuser interface 128 to directly control the home network server 120, toregister or remove the home devices 110-1 to 110-n in/from the homenetwork server 120, or to control the home devices 110-1 to 110-n viathe home network server 120.

The network interface module 127 may be, for example, an Internetcommunication module, and may connect the home network server 120 to theexternal network.

The storage unit 129 may be configured to store, under control of thecontroller 121, a program code, data or information used for operationof the home network server 120. The storage unit 129 may store a largeamount of data provided from the external devices or the home devices110-1 to 110-n.

FIG. 4 is a block diagram illustrating a configuration of a messengerserver according to an embodiment of the present disclosure.

Referring to FIG. 4, the messenger server 200 may include acommunication module 210, a controller 220, a user DataBase (DB) 230, ahome device DB 240, and a command DB 250.

The communication module 210 may be configured to communicate with themobile terminal 300 via the wireless network 130 b, and may beconfigured to communicate with the home network server 120 via the wirednetwork 130 a. The communication module 210 may receive a chat messagefrom the mobile terminal 300 via the wireless network 130 b, and send acontrol message corresponding to the chat message to the home networkserver 120 via the wired network 130 a.

The controller 220 may control and manage the overall operation of themessenger server 200. The controller 220 may manage and control the userDB 230 storing user information of the mobile terminal 300, and the homedevice DB 240 storing home device information of the home device that isregistered for the mobile terminal 300 or the user. In addition, thecontroller 220 may manage and control the command DB 250 storing avariety of control commands in the form of a chat message, which wereregistered in advance by the user to control the home device.

FIG. 5 is a block diagram illustrating a configuration of a mobileterminal according to an embodiment of the present disclosure. For abetter understanding of the description, reference will be made to FIGS.1 to 4.

Referring to FIG. 5, the mobile terminal 300 may include at least one ofa controller 310, a cellular communication module 320, asub-communication module 322, a multimedia module 328, a camera module336, a Global Positioning System (GPS) module 338, an Input/Output (I/O)module 344, a sensor module 342, a storage unit 358, a power supply unit340 and a display 360.

The camera module 336 and the GPS module 338 may be optional. Thesub-communication module 322 may include at least one of a WLAN module324 and a short-range communication module 326. The multimedia module328 may include at least one of a broadcasting & telecommunicationsmodule 330, an audio playback module 332 and a video playback module334. The camera module 336 may include at least one camera, and the I/Omodule 344 may include at least one of a button set 346, a microphone(MIC) 348, a speaker (SPK) 350), a vibration motor 352, a connector 353,and a keypad 356.

The controller 310 may include a CPU 310-1, a ROM 310-2 storing acontrol program for controlling the overall operation of the mobileterminal 300, and a RAM 310-3 that temporarily stores the signals ordata received from the outside of the mobile terminal 300, or is used asa memory area for operations performed in the mobile terminal 300.

The cellular communication module 320, under control of the controller310, may connect the mobile terminal 300 to an external device (e.g., abase station of the cellular system) via at least one or multipleantennas (not shown), using wireless access technology based on thecellular communication protocol. The cellular communication module 320may transmit and receive radio signals containing voice calls, videocalls, Short Messaging Service (SMS) messages or Multimedia MessagingService (MMS) messages, to/from other devices capable of communication.The cellular communication module 320 may transmit and receive radiosignals containing short data for control of an application and thelike.

The WLAN module 324, under control of the controller 310, may access theInternet in the place where a wireless Access Point (AP) (not shown) isinstalled. The short-range communication module 326, under control ofthe controller, may perform wireless short-range communication betweenthe mobile terminal 300 and the external devices. The type ofshort-range communication may include Bluetooth, ZigBee, Infrared DataAssociation (IrDA), and the like.

The display 360 may display information about various applications(e.g., call application, data transfer application, broadcastapplication, camera application and the like) including an instantmessenger application executable by the controller 310, and may includea touch screen providing user interfaces configured to be adapted to theapplications. The controller 310 may select a soft key displayed on thedisplay 360 in response to a user gesture detected on the display 360,or may execute an application or function corresponding to the soft key.The user gesture may include a touch by a finger or a tool, and motionrecognition by the human body.

The instant messenger application may be installed in the ROM 310-2 orthe storage unit 358 of the mobile terminal 300, and if the instantmessenger application is executed, the instant messenger application mayprovide various user interfaces (or user interface screens) as shown inFIGS. 10B to 10G and FIGS. 11A and 11B, which will be described indetail below, generate a control command for controlling an operation ofthe home device 110 (in FIG. 1) in response to the user's key input ortouch input which is entered to the user interface, or to a gestureinput that is made using an acceleration sensor, a gyro sensor or thelike, and send the control command to the messenger server 200 (in FIG.2).

The storage unit 358 may store the operating status information andoperation processing result information of the home devices 110, whichare provided from the home network server 120 via the messenger server200 (in FIG. 2) in response to the various applications, content anddata, and to the remote control of the present disclosure.

FIG. 6 is a block diagram illustrating a configuration of a home networksystem for remotely controlling operations of home devices according toan embodiment of the present disclosure.

In the embodiment of FIG. 6, the home network server 120 may registerinformation about multiple home devices 140, collect the operatingstatus information or information on the operation processingcorresponding to the operations of the home devices 140 periodically orbased on a request, and check the operating status or operationprocessing results of each of the home devices 140.

As for the multiple home devices 140, their locations may be registeredindividually for each room in the home. In FIG. 6, reference numerals140-1 to 140-n each represent at least one home device located in eachroom.

The home network server 120 may receive a control message forcontrolling the home devices 140 from the messenger server 200 via thewired network 130 a, and transmit the operating status information ofthe home devices 140 that are controlled in response to the controlmessage, to the mobile terminal 300 via the wireless network 130 b.

The home network server 120 may store location information indicatingdevice locations for each room in the home, and register and manage thelocations of the home devices 140 for each room. The locationinformation may be provided to the home network server 120 from a server(not shown) of the operator that builds the home network system, or maybe the information that the user has directly input to the home networkserver 120 using a personal computer capable of accessing the wirednetwork 130 a. As for the location information, if the user's homeaddress is entered through the portal site provided by the operator, thelocation information may be provided in a form including floor plans ofa variety of designs, which are selective or editable.

The user may directly register the current locations of the home devices140 in the home network server 120, or may detect the current locationsof the home devices 140 using a Radio Frequency (RF) tag, a sensor andthe like mounted in each of the home devices 140. The locationinformation of the home devices 140 may be mapped to names of the roomswhere the home deices 140 are located. For example, a refrigerator maybe mapped to the kitchen, and a TV may be mapped to the living room. Asfor the location information, each home device may have informationabout the location where the home device is generally installed, as adefault value.

In the home network system of FIG. 6, the home network server 120 mayregister information about the home devices 140 installed in the home,and control and manage the registered home devices 140. For example, ifa new home device is installed, the home network server 120 may collectand store information about the new home device (e.g., identifier,capacity information and operating status information of the homedevice). The information about the home device may be directly obtainedfrom the home device 140 through a service discovery procedure betweenthe home network server 120 and the home device 140, or may betransmitted to the home network server 120 from an Internet server basedon the identification information provided from the home device 140, ormay be directly input to the home network server 120 by the user or thesystem administrator. If the user purchases and installs a new homedevice, and inputs, to the home network server 120, identificationinformation (e.g., a serial number, a model number, a Quick Response(QR) code and the like) provided through the home device or the user'sguide for the home device, the home network server 120 may access themanufacturer's server over the Internet based on the identificationinformation of the home device to obtain information about the homedevice.

The information about the home devices 140 registered in the homenetwork server 120 may be registered in the same way even in themessenger server 200. In other words, the user may transmit homedevice-related information (e.g., identification information of a homedevice, in-home location information of each home device, and the like)to the messenger server 200 using the personal computer capable ofaccessing the wired network 130 a. As for the in-home locationinformation of each home device, among the home device-relatedinformation, if the user's home address is entered through the portalsite provided by the operator of the messenger server 200, the locationinformation may be provided in a form including floor plans of a varietyof designs, which are selective or editable. The home device-relatedinformation provided to the messenger server 200 may be stored in thehome device DB 240 (in FIG. 4) interworking with the messenger server200.

FIG. 7 is a block diagram illustrating a configuration of a home networksystem for remotely controlling operations of home devices using aninstant messenger according to an embodiment of the present disclosure.

In the example of FIG. 7, the home network system may control anoperation of a washer 141 located in one room (e.g., a laundry room)through a messenger interface provided by an instant messengerapplication, and may also control an operation of a robot vacuum cleaner147 located in another room (e.g., a living room).

It will be assumed in FIG. 7 that the washer 141, a Closed-CircuitTelevision (CCTV) 143 and an electric lighting 145 are installed in aroom (e.g., the laundry room) as fixed home devices 140 a. The fixedhome devices 140 a may be connected to the home network server 120through a wired interface or a wireless interface. It will be assumedthat the robotic vacuum cleaner 147 having a moving function is locatedin another room (e.g., the living room) as a mobile home device. Therobotic vacuum cleaner 147 may be connected to the home network server120 via a wireless interface.

In the example of FIG. 7, for operation control of the washer 141, acontrol command in the form of a chat message that is entered through anexecuted instant messenger application may be sent to the messengerserver 200 through the mobile terminal 300, and the messenger server 200may search the command DB 250 (in FIG. 4) for a control message mappedto a control command in the form of a chat message, and send the foundcontrol message to the home network server 120. The home network server120 may control an operation of the washer 141 in response to thecontrol command included in the received control message. The homenetwork server 120 may also control an operation of the robotic vacuumcleaner 147 in the same way.

FIG. 8 is a flow diagram illustrating a registration process ofregistering a home device in a home network server according to anembodiment of the present disclosure.

Referring to FIG. 8, a home device 140 may be installed and powered on,in operation 801.

In operation 803, the home network server 120 may perform a discoveryprocedure for the home device 140. For example, if the home device 140is connected to the home network server 120 by WiFi, the home networkserver 120 may perform a discovery procedure for a nearby new homedevice periodically or at the request of the user. Upon detecting thehome device 140 through the discovery procedure, the home network server120 may receive home device information from the detected home device140. The home device information may include at least one ofidentification information and performance information of the homedevice 140. Optionally, the identification information may include aproduct code indicating the product type (e.g., an air conditioner, arefrigerator, an LED lighting, and the like) of the home device 140.

Optionally, in operation 805, the home network server 120 may receive aregistration request for the home device 140 that the user desires toregister, directly from the user via a computer over the Internet, andmay also receive information about the home device 140 from the user. Asanother example, the home network server 120 may obtain identificationinformation for the home device 140 from the home device 140 or theuser, and access the manufacturer's server for the home device 140 basedon the identification information to obtain performance informationabout the home device 140.

In operation 807, the home network server 120 may store informationabout the home device 140, and manage the home device 140 that isregistered using the stored information about the home device 140.Optionally, the home network server 120 may receive additionalinformation about the home device 140 directly from the user, or from aremote terminal over the Internet, WiFi or the like. The additionalinformation may include, for example, at least one of the name ornickname of the home device 140, and the location where the home device140 is installed.

Optionally, the home network server 120 may receive additionalinformation about an operation mode and/or rights of the home device 140directly from the user, or from a remote terminal over the Internet,WiFi or the like. The additional information may include, for example, alevel of the access rights to the home device 140. The level of theaccess rights may include, for example, (i) allowance of the access onlyvia the home network server 120, (ii) allowance of the access by theremote terminal registered in the messenger server 200, (iii) allowanceof the access by the remote terminal registered in the home networkserver 120, or the like.

In operation 809, the home network server 120 may set up its connectionto the home device 140. The setup of the connection may be performed bya procedure determined in accordance with a WiFi, a ZigBee, a Bluetooth,an NFC or a wired communication scheme, depending on the communicationscheme between the home network server 120 and the home device 140. If aconnection is set up between the home network server 120 and the homedevice 140, the home device 140 may transmit data for the home device140 to the home network server 120 in operation 811.

In operation 813, the home network server 120 may send a control commandto the home device 140. Optionally, the control command may be setconsidering the product type of the home device 140, to meet thecharacteristics of the product. The control command may be set to keepthe current operating status, or to change the operating status step bystep. The control command may be set in common for multiple home devices140, or different control commands may be set for the multiple homedevices 140. Although it will be described in detail below, the controlcommand may be sent from the messenger server 200 to the home networkserver 120, and the home network server 120 may forward the controlcommand to the home device 140.

FIG. 9 is a flow diagram illustrating a process of registering a homedevice in a messenger server through an instant messenger applicationand a process of remotely controlling the registered home deviceaccording to an embodiment of the present disclosure.

Referring to FIG. 9, the mobile terminal 300 may execute or run aninstant messenger application (or a messenger application) for remotecontrol of a home device, in operation 901. Optionally, the mobileterminal 300 may further perform an operation of installing themessenger application.

In operation 903, the mobile terminal 300 may send a registrationrequest message for the home device 140 to the messenger server 200through the executed instant messenger application. The registrationrequest message may include, for example, information about the homedevice 140 registered in the home network server 120, a control commandin the form of a chat message, which is for controlling the home device140, information about the mobile terminal 300, user authenticationinformation, and the like.

The information about the home device 140 may include at least one ofidentification information and performance information of the homedevice 140. Optionally, the identification information may include aproduct code indicating the product type (e.g., an air conditioner, arefrigerator, an LED lighting, and the like) of the home device 140. Thecontrol command in the form of a chat message may be a text that theuser of the mobile terminal 300 has arbitrarily set. For example, thecontrol command may be a text in the form of a universal chat message,which is exchanged between users in the chat environment, such as “poweron”, “power off”, “turn up the temperature”, “turn down thetemperature”, “turn on the lighting”, “turn off the lighting”, and thelike. As another example, the control command may be a text in the formof a chat message, which can simultaneously control home devicesexisting in a specific location in the home, or home devices that can begrouped in the same category, such as “turn on the lighting in thebathroom”, “turn off the lighting in the living room”, “turn off thelightings #1, #2 and #3 in the living room”, “turn on all the rights inrooms #1, #2 and #3”, and the like.

The information about the mobile terminal 300 may be the userinformation that is registered in advance to use the messenger service.For example, the information about the mobile terminal 300 may includethe user name registered during subscription to the messenger service,the phone number of the mobile terminal, the ID and password set by theuser, and the like.

In operation 905, the messenger server 200 may receive the home deviceinformation from the home network server 120 over the wired network 130a. In operation 907, the messenger server 200 may determine whetherthere is information matching the home device information included inthe registration request message from the mobile terminal 300 among thehome device information received from the home network server 120, andif there is the matching information, the messenger server 200 may storethe home device information and the control command in the form of achat message, which the user has set for the home device, in the homedevice DB 240 (in FIG. 4) and the command DB 250 (in FIG. 4),respectively, completing the registration.

In operation 909, the messenger server 200 may send a registrationapproval message indicating the successful registration of the homedevice information, to the mobile terminal 300.

Thereafter, the mobile terminal 300 may send a control command forremotely controlling the home device to the home device, using theinstant messenger application.

For example, upon receiving in operation 911 a text command through achat window that the instant messenger application provides to remotelycontrol the home device, the mobile terminal 300 may send the textcommand to the messenger server 200 as a chat message in operation 913.

The messenger server 200 may generate a control message corresponding tothe chat message with reference to the command DB 250 (in FIG. 4) inoperation 915, and send the control message to the home network server120 in operation 917.

In operation 919, the home network server 120 may send a control commandcorresponding to the control message to the home device 140 through thewired/wireless interface. The home device 140 may change its currentoperating status in response to the control command. In operations 921and 923, the changed operating status information may be transmitted tothe messenger server 200 via the home network server 120. In operation925, the messenger server 200 may transmit the operating statusinformation to the mobile terminal 300 as a reply chat message.

FIGS. 10A to 10G illustrate examples of various user interface screensdisplayed on a mobile terminal during execution of remote controlaccording to an embodiment of the present disclosure.

Referring to FIGS. 10A to 10G, if the user desires to remotely drive orcontrol the home device 140, the user may execute or run the messengerapplication for remote control of the home device 140, on the mobileterminal 300. To this end, as illustrated in FIG. 10A, the mobileterminal 300 may provide a messenger icon 1001 for execution of aninstant messenger application. Upon detecting a user gesture (e.g., atouch) for the messenger icon 1001, the mobile terminal 300 may executethe instant messenger application.

If the instant messenger application is executed, the mobile terminal300 may provide a chat list screen registered by the user as illustratedin FIG. 10B. In a chat list displayed on the chat list screen may bedisplayed a ‘home network server’ item 1003 together with a friend listmade up of the user's friends. If the user selects the ‘home networkserver’ item 1003, the mobile terminal 300 may provide a list of homedevices that the user has registered in advance, as illustrated in FIG.10C. In other words, the list of home devices may be configured to forma sub-category of the ‘home network server’ item 1003. In an alternativeembodiment, the friend list and the home device list may be displayedtogether in the chat list of FIG. 10B.

In this embodiment, remote control of a washer will be considered. Ifthe user selects a ‘washer’ item 1005 in FIG. 10C, a chat type selectionscreen for asking a chat type for the selected washer may be provided asillustrated in FIG. 10D. On the chat type selection screen may beprovided an icon 1007 in the form of a washer, and a ‘1:1 chat’ menu1009 and a ‘group chat’ menu 1011 disposed under the icon 1007.

If the ‘1:1 chat’ menu 1009 is selected in response to a user input, achat window including soft keys 1013 may be provided as illustrated inFIG. 10E. If a text such as ‘power on’ 1015 is input using the soft keys1013, the mobile terminal 300 may provide (or display) a text command of‘power on’ 1015 in the chat window, generate a chat message in the formof a packet at the same time using the text command and theidentification information indicating the washer, and send the chatmessage to the messenger server 200 over the communication network. Inresponse, the messenger server 200 may generate a control message mappedto the text command included in the received chat message with referenceto the command DB 250 (in FIG. 4), and send the control message to thehome network server 120. Based on the identification information of thewasher, which is included in the received control message, the homenetwork server 120 may determine that the control target is a washer andthe control message corresponds to a control command to power on. Then,the home network server 120 may send, to the washer, a control commandto turn on power of a washer.

The home network server 120 may receive, from the washer, the results ofprocessing (e.g., operating status information of the washer) performedin response to the control command, and transmit the operating statusinformation to the messenger server 200. The messenger server 200 mayconvert the operating status information into a format of a chatmessage, and send the chat message to the mobile terminal 300. Forexample, if the washer is powered on while it was powered off, themessenger server 200 may send a chat message indicating completion ofpower-on to the mobile terminal 300. The mobile terminal 300 may providea chat message such as ‘power-on completed’ 1017 to the user asillustrated in FIG. 10E. By providing the chat message to the user, themobile terminal 300 may allow the user to check the processing resultsfor the control message that the mobile terminal 300 sent in the form ofa chat message.

Instead of directly inputting the text command in the chat window usingthe soft keys as illustrated in FIG. 10E, the user may input his/herdesired text command using pre-registered text menus 1014, 1016, 1018and 1020 as illustrated in FIG. 10F. In other words, the user may selecta text menu by a touch input corresponding to his/her desired textcommand from among the text menus 1014, 1016, 1018 and 1020, to input atext command. With the use of the touch input, the user may input adesired text command by simply selecting a desired text menu, withoutdirectly inputting the text command using the soft keys. For example, ifthe user selects a message menu of ‘power on’ 1014, a chat message of‘power on’ 1023 may be automatically displayed in a chat window 1100,and at the same time, the chat message may be sent to the messengerserver 200. In the subsequent operation, as described in FIG. 10E, ifthe home network server 120 receives the operating status information ofthe washer and transmits the operating status information to themessenger server 200, the messenger server 200 may convert the operatingstatus information into a chat message and send the chat message to themobile terminal 300.

As described in FIG. 10E, if the operating status of the washer isswitched from a power-off state to a power-on state, the mobile terminal300 may provide a chat message such as ‘power-on completed’ 1025 to theuser.

Similarly, if the user selects the ‘start laundry’ menu 1016, a chatmessage 1027 corresponding to the ‘start laundry’ menu 1016 may beautomatically displayed in the chat window, and at the same time, thechat message may be sent to the washer in the form of a packet.

The washer may send the laundry progress situation such as ‘laundrystarted’ 1029, ‘laundry in progress’ 1031, ‘laundry completed’ 1033 andthe like to the mobile terminal 300 via the home network server 120 andthe messenger server 200 in real time as a reply chat message. If themobile terminal 300 receives the chat message of ‘laundry completed’1033, the user may select a text menu command such as ‘startdehydration’ 1018. At which point, a chat message of ‘start dehydration’1035 is sent and then a chat message of ‘dehydration started’ 1037 isreceived to/from the washer. If the washer completes the dehydration,the mobile terminal 300 may receive a chat message of ‘dehydrationcompleted’ 1039. If all the laundry is terminated, the user may select atext menu command of ‘power off’ 1020 and send a chat message 1041corresponding to the text menu command to the washer. In response, thewasher may turn off its power, and if the power-off is completed, areply chat message of ‘power-off completed’ may be sent to the mobileterminal 300. In this way, the user may check the laundry progresssituation from the laundry start to the power-off in real time, usingthe reply chat message.

If the washer is in an inoperable state where it cannot execute theuser's control command, or is in a mal-operation state, the home networkserver 120 may inform the messenger server 200 of the washer'sinoperable or mal-operation state. Then, the messenger server 200 mayconvert the home device's inoperable or mal-operation state into a chatmessage, and send the chat message to the mobile terminal 300.

Although a washer is remotely controlled using a chat message in theexample of FIGS. 10A to 10G, it will be apparent to those of ordinaryskill in the art that other types of home devices may also be remotelycontrolled using a chat message. For example, in a case where the userdesires to remotely control a vacuum cleaner, if the user directlyinputs a text command such as ‘drive cleaner’, ‘start cleaning’ or thelike as in FIG. 10E, or if the text command is received through a textmenu command as in FIG. 10F, the messenger server 200 may receive a chatmessage corresponding to the text command or the text menu command, andsend a control command corresponding to the chat message to the homenetwork server 120.

The text menu commands 1014, 1016, 1018 and 1020 in FIG. 10F may beactivated in the form of a pop-up window. For example, if the userselects a specific key among the soft keys 1013 in FIG. 10E, the textmenu commands 1014, 1016, 1018 and 1020 in FIG. 10F may be provided inthe form of a pop-up window 1050, as illustrated in FIG. 10G. Byselecting the text menu commands provided in the form of the pop-upwindow 1050, the user may easily input text commands without directlyinputting the text commands using the soft keys.

FIGS. 11A and 11B illustrate examples of user interface screensdisplayed on a mobile terminal during execution of remote controlaccording to another embodiment of the present disclosure.

While FIGS. 10A to 10G show an embodiment of remotely controlling onehome device in a 1:1 chat message, FIGS. 11A and 11B show anotherembodiment of remotely controlling grouped multiple home devices at thesame time. In this embodiment, the ‘group chat’ menu 1011 like in FIG.10D may be provided to simultaneously control the grouped multiple homedevices. The ‘group chat’ menu 1011 may be useful when there is a needto control in common the home devices capable of grouping. The term‘capable of grouping’ as used herein may refer to the feasibility ofremotely controlling in common multiple home devices with one controlcommand. In other words, the home devices capable of grouping may bedefined as home devices of the same type. The home devices capable ofgrouping may include, for example, a lighting, an air conditioner or thelike, which are installed in each room, the living room and theentrance.

If the user selects ‘lighting’ from the chat list in FIG. 10C, and then,selects the ‘group chat’ menu 1011 in FIG. 10D, a pre-registeredlighting list may be provided as illustrated in FIG. 11A. A home devicelist provided through the ‘group chat’ menu 1011 may be provided in thecombined form of the in-home location information (e.g., room 1, room 2,room 3, living room, and entrance) and the home devices' name (e.g.,lighting) as illustrated in FIG. 11A.

In the lighting list, if the user desires to remotely control ‘lightingof room 1’ 1101, ‘lighting of room 3’ 1103 and ‘lighting of living room’1105 in a group chat message, the user may make a check mark on items1107, 1109 and 1111 that the user desires to remotely control, and then,select ‘complete’ 1112, thereby causing the ‘lighting of room 1’ 1101,the ‘lighting of room 3’ 1103 and the ‘lighting of living room’ 1105 toparticipate in the group chat as illustrated in FIG. 11B.

As described in FIG. 10E, if the user directly inputs a predefined chatmessage such as ‘turn on lighting’ 1114 using the soft keys, the chatmessage may be sent to the messenger server 200, and the messengerserver 200 may generate a control message mapped to the chat message,and send the control message to the home network server 120. The homenetwork server 120 may check the identification information (e.g.,lighting of room 1, lighting of room 3, and lighting of living room) ofthe lightings, the location information (e.g., room 1, room 3, andliving room) of the lightings, and the control command for controllingthe lightings in common, all of the identification information, thelocation information and the control command being included in thecontrol message, and then, send the checked control command (e.g., ‘turnon lighting’) 1114 to each of the ‘lighting of room 1’ 1101, the‘lighting of room 3’ 1103 and the ‘lighting of living room’ 1105.

Each of the ‘lighting of room 1’ 1101, the ‘lighting of room 3’ 1103 andthe ‘lighting of living room’ 1105 may transmit the operating statusinformation corresponding to the control command to the messenger server200 via the home network server 120, and the messenger server 200 maygenerate a reply chat message (e.g., ‘turn on lighting’) indicating theoperating status information, and send the reply chat message to themobile terminal 300. The mobile terminal 300 may provide or displayreply chat messages 1116, 1118 and 1120 received from the associatedlightings in the chat window, for the user.

As described above, the present disclosure allows the user to remotelycontrol a home device using an instant messenger application universallyinstalled in the user's mobile terminal, without the need to install aseparate application for controlling home devices. In addition, the usermay easily remotely control a home device using the friendly andfamiliar UI environment provided by the instant messenger application.

As is apparent from the foregoing description, according to anembodiment of the present disclosure, there is no need to install aseparate application for controlling home devices in the mobileterminal. In addition, the user may control the home device using theinstant messenger application universally installed in the mobileterminal, so the user may easily remotely control the home device usingthe friendly and familiar UI environment provided by the instantmessenger application.

For example, although a control command (or a control message) that isconverted from a chat message by the messenger server 200 is sent to thehome device via the home network server 120 in the above-describedembodiments, the control command (or the control message) may bedirectly sent to the home device, without passing though the homenetwork server 120. In addition, the home network server 120 may beimplemented as a home gateway that is connected to some of the homedevices in the home. In this case, the messenger server 200 may send thecontrol command (or the control message) to some of the home devices viathe home gateway, and may directly send the control command (or thecontrol message) to the remaining home devices.

While the present disclosure has been shown and described with referenceto various embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present disclosure asdefined by the appended claims and their equivalents.

What is claimed is:
 1. A method for remotely controlling a home devicein a mobile terminal, the method comprising: transmitting, to a server,a request message for registering the home device using a chat window ofa chat application; receiving, from the server, a response messageindicating that the home device is registered to the server; receiving achat message input to the chat window, the chat message being associatedwith a control of the registered home device; and transmitting the chatmessage to the server; wherein the transmitted chat message isconverted, at the server, into a corresponding control command, which istransmitted to the home device.
 2. The method of claim 1, furthercomprising: receiving a reply chat message indicating a result for thechat message from the server; and displaying the reply chat messagethrough the chat window.
 3. The method of claim 2, wherein the chatmessage includes at least one of a 1:1 chat message that is mapped to acontrol command for controlling one home device, and a group chatmessage that is mapped to a group control command for simultaneouslycontrolling at least two grouped home devices.
 4. The method of claim 1,wherein the chat message includes at least one of identificationinformation of the home device, in-home location information of the homedevice, control information for controlling an operation of the homedevice, and user information of the mobile terminal.
 5. The method ofclaim 1, wherein the chat message includes at least one of messagesregistered in advance.
 6. The method of claim 1, wherein the chatmessage includes any one text menu selected from multiple text menus. 7.The method of claim 6, wherein the multiple text menus are providedthrough a pop-up window.
 8. A method for remotely controlling a homedevice in a server communicating with a mobile terminal, the methodcomprising: receiving a request message for registering at least onehome device, from a mobile terminal; discovering the home device basedon the request message; receiving information of the home device;registering the home device in the server; transmitting a responsemessage indicating that the home device is registered to the server, tothe mobile terminal; receiving, from the mobile terminal, a chat messageinput to a chat window in a chat application of the mobile terminal, thechat message being associated with a control of the registered homedevice; converting the chat message into a corresponding controlmessage; and transmitting the converted control message to the at leastone home device.
 9. The method of claim 8, wherein the control messageis agreed between the server and the home device.
 10. The method ofclaim 8, further comprising: receiving an operation processing resultfor the control message from the home device; converting the chatmessage into a reply chat message indicating the received operationprocessing result; and transmitting the reply chat message to the mobileterminal.
 11. The method of claim 8, wherein the information of the homedevice is received from at least one of the mobile terminal andmanufacturer server of the home device.
 12. A mobile terminal forremotely controlling a home device, the mobile terminal comprising: astorage unit configured to store an execution file of a chatapplication; a controller configured to execute the execution file ofthe chat application; and a transceiver configured to transmit, to aserver, a request message for registering the home device using a chatwindow of a chat application, to receive, from the server, a responsemessage indicating that the home device is registered to the server, toreceive a chat message input to the chat window, the chat message beingassociated with a control of the registered home device, and to transmitthe chat message to the server, wherein the transmitted chat message isconverted, at the server, into a corresponding control command, which istransmitted to the home device.
 13. The mobile terminal of claim 12,wherein the transceiver receives a reply chat message indicating aresult for the chat message from the server and the reply chat messageis displayed through the chat window.
 14. The mobile terminal of claim13, wherein the chat message includes at least one of a 1:1 chat messagethat is mapped to a control command for controlling one home device, anda group chat message that is mapped to a group control command forsimultaneously controlling at least two grouped home devices.
 15. Themobile terminal of claim 12, wherein the chat message includes at leastone of identification information of the home device, in-home locationinformation of the home device, control information for controlling anoperation of the home device, and user information of the mobileterminal.
 16. The mobile terminal of claim 12, wherein the chat messageincludes at least one of messages registered in advance.
 17. The mobileterminal of claim 12, wherein the chat message includes any one textmenu selected from multiple text menus.
 18. The mobile terminal of claim17, wherein the multiple text menus are provided through a pop-upwindow.
 19. A server for remotely controlling a home device, the servercommunicating with a mobile terminal, the method comprising: atransceiver configured to receive a request message for registering atleast one home device, from a mobile terminal; a processor configured todiscover the home device based on the request message, wherein thetransceiver configured to receive information of the home device and totransmit a response message indicating that the home device isregistered to the server, to the mobile terminal, wherein the processorconfigured to registering the home device in the server.
 20. The serverof claim 19, wherein the control message is agreed between the serverand the home device.
 21. The server of claim 19, further comprising:wherein the transceiver configured to receive an operation processingresult for the control message from the home device, wherein theprocessor configured to convert the chat message into a reply chatmessage indicating the received operation processing result, wherein thetransceiver configured to transmit the reply chat message to the mobileterminal.
 22. The server of claim 19, wherein the information of thehome device includes at least one of identification information of thehome device, capability information of the home device, a device codeinformation identifying the home device, a nickname of the home deviceinformation and location information of the home device.
 23. The serverof claim 19, wherein the information of the home device is received fromat least one of the mobile terminal and manufacturer server of the homedevice.